Ophthalmic statistics note 1: unit of analysis

SCENARIO 1 A senior colleague has conducted a study and asks me to analyse their data. They give me an excel spreadsheet containing 80 observations of intraocular pressure (IOP)—40 of which were made after using drug A and 40 of which were made after using drug B (see table 1). My colleague asks me to find out whether drug A or drug B is better in terms of reducing IOP. I think back to my medical school training and recall something called a t test, which I believe may be relevant. Using the internet I discover that there is indeed a t test and that the t test can be used to compare the means of two groups. I apply the test and feedback the results (including a two-tailed p value of 0.0024) to my colleague (see online supplementary appendix 1, figure S1 for results of analysis). The paper is written and submitted. After some time, it is returned with several comments made by reviewers. One of these questions is whether I have assessed the assumptions made by a t test. I must admit to not being entirely clear what this sentence means. Using the internet once more, I learn that the p value I quoted to my colleague has been calculated using statistical theory, but that if this p value is to be regarded as robust or valid, my data must adhere to some rules (or assumptions). Assumption 1 of the t test is that my data follow approximate normality. I draw a histogram of my IOP observations and see that it looks roughly symmetric, which reassures me somewhat as evidently rough normality can be assumed with an approximate symmetric histogram (see figure 1). Assumption 2 is that the data points are independent. I am not familiar with the term independent in this context, but using the internet I find that statistically independent means no relationship between data points. I have 40 IOP observations made on drug A and 40 made on drug B. I email my senior colleague to check that the observations are not made on the same subjects—for example, the 80 measurements might actually relate to measurements on 40 patients each treated with both drug A and drug B. If this is the case, then I have measures of IOP after treatments A and B on the same patients and it seems clear to me that there will be a relationship between IOP measurements made on the same patient. My colleagues’ response comes with good and bad news. I am reassured that the subjects treated with drug A are not the same as those treated with drug B. I am also told, however, that there are actually only 10 patients on drug A and 10 on drug B. Further discussion reveals that my dataset consists of two observations on the right eye and two observations on the left eye of each subject (see table 2). Suddenly my dataset has revealed a complication that I simply hadn’t considered. There are multiple observations and related observations. Clearly independence isn’t adhered to... (see online supplementary appendix 1, figure S2 for results of analysis of the mean IOP in either the left or the right eye).